✨ `dynamic-sidecar` allows to import old state from legacy services 🚨 #7675

GitHK · 2025-05-14T11:36:02Z

What do these changes do?

Allows to define services with the new LegacyState attribute inside PathMappingsLabel which maps the state from the legacy version to the state in the new style version of the service.

Example Jupyter FEniCS:

legacy workspace
new style workspace (pleas note the inputs and output have been copied over)

Related issue/s

How to test

Dev-ops

🚨 @GitHK releases a new version of https://github.com/ITISFoundation/ci-service-integration-library when this PR is merged in master

codecov · 2025-05-14T14:08:26Z

Codecov Report

Attention: Patch coverage is 72.15190% with 22 lines in your changes missing coverage. Please review.

Project coverage is 88.96%. Comparing base (3595349) to head (9f266a5).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7675      +/-   ##
==========================================
+ Coverage   87.53%   88.96%   +1.43%     
==========================================
  Files        1811     1453     -358     
  Lines       70260    60117   -10143     
  Branches     1137      470     -667     
==========================================
- Hits        61500    53483    -8017     
+ Misses       8451     6514    -1937     
+ Partials      309      120     -189

Flag	Coverage Δ
integrationtests	`64.41% <32.25%> (+0.04%)`	⬆️
unittests	`88.08% <72.15%> (+1.32%)`	⬆️

Components	Coverage Δ
api	`∅ <ø> (∅)`
pkg_aws_library	`∅ <ø> (∅)`
pkg_dask_task_models_library	`∅ <ø> (∅)`
pkg_models_library	`93.06% <93.75%> (-0.03%)`	⬇️
pkg_notifications_library	`∅ <ø> (∅)`
pkg_postgres_database	`∅ <ø> (∅)`
pkg_service_integration	`69.92% <ø> (ø)`
pkg_service_library	`∅ <ø> (∅)`
pkg_settings_library	`∅ <ø> (∅)`
pkg_simcore_sdk	`85.07% <15.00%> (-0.54%)`	⬇️
agent	`96.46% <ø> (ø)`
api_server	`91.62% <ø> (ø)`
autoscaling	`96.08% <ø> (ø)`
catalog	`92.64% <ø> (ø)`
clusters_keeper	`99.25% <ø> (ø)`
dask_sidecar	`89.86% <ø> (ø)`
datcore_adapter	`98.12% <ø> (ø)`
director	`76.80% <ø> (ø)`
director_v2	`91.12% <100.00%> (-0.03%)`	⬇️
dynamic_scheduler	`96.76% <ø> (ø)`
dynamic_sidecar	`90.18% <77.77%> (-0.04%)`	⬇️
efs_guardian	`89.79% <ø> (ø)`
invitations	`93.28% <ø> (ø)`
payments	`92.63% <ø> (ø)`
resource_usage_tracker	`89.18% <ø> (+0.16%)`	⬆️
storage	`87.63% <ø> (-0.11%)`	⬇️
webclient	`∅ <ø> (∅)`
webserver	`85.82% <ø> (+<0.01%)`	⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3595349...9f266a5. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…or-new-services

Copilot

Pull Request Overview

This PR introduces support for importing and migrating legacy service state by adding a new LegacyState model, integrating it into configuration, workflows, and environment specs.

Adds a LegacyState Pydantic model and validation in PathMappingsLabel
Extends dynamic-sidecar settings and long-running tasks to handle legacy state paths
Propagates DY_SIDECAR_LEGACY_STATE through Docker environment variables and SDK data manager operations

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
services/dynamic-sidecar/.../core/settings.py	Adds `DY_SIDECAR_LEGACY_STATE` configuration field
services/dynamic-sidecar/.../modules/long_running_tasks.py	Implements `_get_legacy_state_with_dy_volumes_path` and passes `legacy_state` to pull/push
services/director-v2/.../docker_service_specs/sidecar.py	Exposes `DY_SIDECAR_LEGACY_STATE` as an environment variable
services/director-v2/tests/unit/.../test_modules_dynamic_sidecar_docker_service_specs.py	Updates expected spec to include `"DY_SIDECAR_LEGACY_STATE": "null"`
services/director-v2/tests/unit/.../test_modules_dynamic_sidecar_docker_service_specs_sidecar.py	Adds `DY_SIDECAR_LEGACY_STATE` to the env var list in sidecar spec tests
packages/simcore-sdk/src/simcore_sdk/node_data/data_manager.py	Adds `legacy_state` parameters to `pull` and `push` functions
packages/simcore-sdk/tests/unit/test_node_data_data_manager.py	Parametrizes `test_pull_legacy_archive` to cover legacy archives
packages/models-library/src/models_library/service_settings_labels.py	Introduces `LegacyState` model, integrates into `PathMappingsLabel`, and adds a validator

Comments suppressed due to low confidence (1)

packages/simcore-sdk/src/simcore_sdk/node_data/data_manager.py:199

There are no unit tests covering the new legacy_destination_path code path in _pull_legacy_archive or the legacy_state branch in push/pull. Consider adding tests for both legacy_state present/absent scenarios.

async def _pull_legacy_archive(

packages/models-library/src/models_library/service_settings_labels.py

services/dynamic-sidecar/src/simcore_service_dynamic_sidecar/modules/long_running_tasks.py

...r-v2/src/simcore_service_director_v2/modules/dynamic_sidecar/docker_service_specs/sidecar.py

packages/models-library/src/models_library/service_settings_labels.py

pcrespov

thx

packages/models-library/src/models_library/service_settings_labels.py

packages/simcore-sdk/src/simcore_sdk/node_data/data_manager.py

…or-new-services

GitHK · 2025-05-16T12:37:13Z

@Mergifyio queue

mergify · 2025-05-16T12:37:26Z

queue

🛑 The pull request has been removed from the queue `default`

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details about the failure.

mergify · 2025-05-16T13:45:17Z

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

❌ unit-tests
☑️ deploy to dockerhub

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

GitHK · 2025-05-19T07:11:29Z

@Mergifyio requeue

mergify · 2025-05-19T07:11:48Z

requeue

✅ This pull request will be re-embarked automatically

The followup queue command will be automatically executed to re-embark the pull request.

mergify · 2025-05-19T07:11:49Z

sonarqubecloud · 2025-05-19T07:14:02Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

mergify · 2025-05-19T07:27:27Z

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

GitHK · 2025-05-19T07:57:56Z

@Mergifyio requeue

mergify · 2025-05-19T07:58:01Z

requeue

✅ This pull request will be re-embarked automatically

The followup queue command will be automatically executed to re-embark the pull request.

mergify · 2025-05-19T07:58:02Z

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at c8bbdc2

fixed issue with get_image_name_and_tag

9c4bb3b

GitHK self-assigned this May 14, 2025

Andrei Neagu added 8 commits May 15, 2025 14:22

extend format

2fc2d1e

passing configuration inside

96ad047

extended

a365ef8

decompressing legacy state path

57d2518

revert cahnge

5a98a78

Merge remote-tracking branch 'upstream/master' into pr-osparc-fixes-f…

3ef3785

…or-new-services

fixed models

2970526

revert

2da2fb8

GitHK changed the title ~~🐛 fixed issue with get_image_name_and_tag~~ dynamic-sidecar is now able to extract the state coming from a different archive when upgrading from a legacy service May 16, 2025

GitHK changed the title ~~dynamic-sidecar is now able to extract the state coming from a different archive when upgrading from a legacy service~~ ✨ dynamic-sidecar allows to import old state from legacy services May 16, 2025

GitHK changed the title ~~✨ dynamic-sidecar allows to import old state from legacy services~~ ✨ dynamic-sidecar allows to import old state from legacy services 🚨 May 16, 2025

Andrei Neagu added 2 commits May 16, 2025 10:13

fixed tests

2cf71e8

mypy

a3442d0

GitHK requested a review from Copilot May 16, 2025 08:16

GitHK added a:dynamic-sidecar dynamic-sidecar service a:simcore-sdk labels May 16, 2025

Copilot AI reviewed May 16, 2025

View reviewed changes

copilot review

7f3182a

GitHK marked this pull request as ready for review May 16, 2025 08:26

GitHK requested review from giancarloromeo, matusdrobuliak66, pcrespov and sanderegg as code owners May 16, 2025 08:26

GitHK requested review from bisgaard-itis and mrnicegyu11 May 16, 2025 08:26

GitHK added this to the Bazinga! milestone May 16, 2025

giancarloromeo reviewed May 16, 2025

View reviewed changes

packages/models-library/src/models_library/service_settings_labels.py Outdated Show resolved Hide resolved

giancarloromeo approved these changes May 16, 2025

View reviewed changes

updated specs

1437db6

GitHK requested a review from mguidon May 16, 2025 09:17

sanderegg approved these changes May 16, 2025

View reviewed changes

pcrespov approved these changes May 16, 2025

View reviewed changes

Andrei Neagu added 3 commits May 16, 2025 14:30

modernized module

5b96536

upgraded tests

32312fb

Merge remote-tracking branch 'upstream/master' into pr-osparc-fixes-f…

616d5bd

…or-new-services

GitHK added the 🤖-automerge marks PR as ready to be merged for Mergify label May 16, 2025

mypy

b58c574

matusdrobuliak66 approved these changes May 16, 2025

View reviewed changes

Merge branch 'master' into pr-osparc-fixes-for-new-services

eb44ffb

Merge branch 'master' into pr-osparc-fixes-for-new-services

9f266a5

mergify bot merged commit c8bbdc2 into ITISFoundation:master May 19, 2025
94 of 96 checks passed

matusdrobuliak66 mentioned this pull request Jun 6, 2025

🚀 Release v1.83.0 #7682

Closed

92 tasks

matusdrobuliak66 mentioned this pull request Aug 5, 2025

🚀 Release v1.84.0 #7947

Closed

88 tasks

✨ dynamic-sidecar allows to import old state from legacy services 🚨 #7675

✨ dynamic-sidecar allows to import old state from legacy services 🚨 #7675

Uh oh!

Conversation

GitHK commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What do these changes do?

Related issue/s

How to test

Dev-ops

Uh oh!

codecov bot commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pcrespov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GitHK commented May 16, 2025

Uh oh!

mergify bot commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛑 The pull request has been removed from the queue default

Uh oh!

mergify bot commented May 16, 2025

Uh oh!

GitHK commented May 19, 2025

Uh oh!

mergify bot commented May 19, 2025

✅ This pull request will be re-embarked automatically

Uh oh!

mergify bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛑 The pull request has been removed from the queue default

Uh oh!

sonarqubecloud bot commented May 19, 2025

Quality Gate passed

Uh oh!

mergify bot commented May 19, 2025

The merge conditions cannot be satisfied due to failing checks:

Uh oh!

GitHK commented May 19, 2025

Uh oh!

mergify bot commented May 19, 2025

✅ This pull request will be re-embarked automatically

Uh oh!

mergify bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ The pull request has been merged automatically

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

✨ `dynamic-sidecar` allows to import old state from legacy services 🚨 #7675

✨ `dynamic-sidecar` allows to import old state from legacy services 🚨 #7675

GitHK commented May 14, 2025 •

edited

Loading

codecov bot commented May 14, 2025 •

edited

Loading

mergify bot commented May 16, 2025 •

edited

Loading

🛑 The pull request has been removed from the queue `default`

mergify bot commented May 19, 2025 •

edited

Loading

🛑 The pull request has been removed from the queue `default`

mergify bot commented May 19, 2025 •

edited

Loading